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^ Class Effects in ATI's 

Selected results from an ATI study a^ee presented in support of the 
importance of Including class analyses ir ATI research. The study 
* nvesti gated the? interactive efn=cts on performance and motivation of 
ftour-motivntion^r aptitudes and two cognitive aptitudes with a choice vs. 
nco-cfooice t> p ea^m©rt iracniipu lattice Both treatments were implemented in 
*^ch of niwie fytrtft or fifth ^raie classruoms (n=165) over a four week 
n^Hod. T(^v^ sets of simple regression results (by treatment, within 
t Uss, and aet^en class) higbTfghted the differential effects of 
{treatment iwdwiidjuals vs. clssses and suggested substantive hypotheses 
out these «f facts. 

Crontrach ancr Snob's (1977) lu - adical rrappraisal of the ATI model" 
o % 9&) ®npBkars?zed the importance of separating between class from within 
zl&sz effects in fffil studies- TThis reappraisal suggested that 
instruction 1 tr<Httments may intp^act with classroom characteristics, as 
well as wjiftifr iividual learner Altitudes. TTtat is, for some 
interact* om *r individual's mentoership in -aawiicular type of classroom 
is tine ttet<*mrS*1ng independent vaf iable, r at^* r than his/her relative 
stamdnnq can afl aptitude characterise. Or . m other instances, 
interasrtiw effects ar^ determined by a co mli^sB ti on of an indiviti* *rs 
standi cm *o ap£itu*ie charac**v 1 stic and reiiev-ant classroom 
characta"iis< K?$. 

Cronb*>h 4*bb (1977) srcc Sustafsson (1978) both investigated 
cla'ss effects ( * i-Hzing anal 55>?s based on class means) in addition to 
overall treetm^n. effects (utiV'ring analyses based' on individuals). In 
both stotfi'eS. sic* 1 i^ictnt interactions in the within treatment analyses 
changed or diVappcartti when class effects were taken into account. These 
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studies indicated that separation of between class and within. class 
effects can contribute to the substantive understanding of interactional 
effects as well as "help track anamolies in the data" (Gustafsson, 1978, 
p. 142). Gustafsson for example, attributed the different results fro;n 
his within treatment and his within class analyses to the particular 
sensitivity of a paired associate aptitude measure to testing conditions. 

Although limited by small sample sizes, the data below are presented 
as further evidence for tha importance of including between class ■ 
analyses in ATT studies. The data presented also suggest an ATI 
methodology by which class effects can be more completely analyzed. In 
ttiis methodology, both treatments are implemented in each classroom, 
rather than assigning whole classes to one treatment (as has been 
customary in ATI research). With this methodology, class x aptitude and 
class x aptitude x treatment interactions are distinguishable. The data 
in this paper represent selected results from\a larger ATI study (Greene, 
1976), the significant components of which are first briefly described. 

, The ATI Study 

Theoretical Framework 

Though softened by time the voices of radical educators (e.g., Kozol, 
1967, 1972), who called so loudly a decade ago for increased student 
choice and control over their own learning activities, can still be 
heard. Moreover, the widespread existence of such educational 
alternatives as open classrooms and self-paced instruction attests to the 



large numbers of people who listened to these voices and accepted what 
they said. However, educational alternatives that incorporate <* 
significant degree of student choice rest on the untested sssuifkM'c ih&t 
all children can make wise educational choices, i.e., choices that 
facilitate the attainment of both cognitive and socioemotior~l 
educational goals (Holt, 1954; Dennison, 1969). The present vfrt*idy was 
designed to test this assumption, using the framework of res ; erare> h Qf* 
aptitude x treatment interactions (ATI), (Cronbach and Snow, *?* r *' M ). 

The ATI framework was selected deliberately since The stud * general 
hypothesis posited an interaction: some children learn better n ^roen 
some choice over the learning situation, while other children 'Sear?, 
better when the learning situation is more structured for Zhm* (Siten 
this general hypothesis, the first step in the design of thii, vMifllrwas 
to identify learner characteristics or aptitudes that would jswtentlie^y^ 
differentiate these two groups of children. More specifics?^, 1 the 
design of the study called for the identification of motivat: "~ 
aptitudes that relate to choice behavior in educational setti 

To this end, a three-phase review of literature was condt fyi 
the first phase, studies investigating aptitudinal^ correlate ^ r 
educational choice were reviewed. The few studies conducted lis area 
provided few ideas" about student aptitudes that may interact h choice 
situations. The research review then shifted to studies im st ating 
the relationship between educational achievement and locus o. - mntrol 
(LOC), which was selected for its logical relationship to fms«%i!ity to 
mafc^choices. This second phase of the review i?rtica4"e^ ttettfetb- LOC- 
achievement relationship is not simple and linear,. buV tpsisad moderated 
by complexities within the LOC construct itself and by other reVsted 



construct!**. These complexities anrl moderating constructs were then 
further investigated in the third phase of the review, which focused on 
two nrajW? areas, within the general body of literature on control and. 
causalt~v learning situations. The first, in the domain of social 
1 eaTraiwr theory, centers around the LOC construct and stems primarily 
front Ifof-'ter (1954). Second, from Heider (1958) comes research relating 
to cau'-^l attributions in the le-ming pTicess. The review of these two 
areas ^ suited -,'n the identification of -four specific otivational 
^Btt.Hua-s that have potentially interacti-ve ---fec f " learning in an 
P^durat-^al choice situation: expectancy of sue • mportance or 
• ncent^ve walue. of success, causal attributions *V academic performance, 

jnd ewMjiative orientation (towards extemallly-'/ ectad vs. 

ielf- 4 tiated learning). (See Greene, 1975 fan* 2 presentation of the 

;oimIe*«? review of literature.) 

Safe^ects . Nine classes of students participated in this study. The 
rrine classes were obtained from four public schools all located withi.i 
-ttre sa»e middle class, predominantly Caucasian, professional community. 
Of the total 165 subjects, 91 were fourth graders and 74 were fifth 
graders^ 

Aptitude measures . An existing instr<i»ent was adapted or a new one 
developed for each of the four motivation* variables identified in the 
review of literature. (Estimates o^ internal consistency reliability for 
each measure were satisfactory, in the .70' s and 80' s.) Measures of two 
cognitive aptitudes were also included to *'sess whether motivational 
apfcitudes had effects on learning beyond thwse of ability (a procedure 
recommended by Cronbach and Snow). i The two cognitive measures were 
pretest performance on the experimental learning task and performance on 
the Lorge-Thorndike Verbal and Nonverbal tesfcs. 



The four motivational measures along with the learning task pretest 
were administered as a pretest package to each class. Performance scores 
on the Lorge-Thomdike tests were attained from school files. 

Learning 1£gtic . The exper iroental learning materi^-V were based on t e 
Thurstone Letter Series task, chosen for its novelty ant£ txs relevance .so 
general probl«» .solving goals of education. These mate— els included a 
Letter Problems Jdorkbook consisr-ng of eight lessons, factr one 
< instructed arosunc a component soil required for the :«ore general 
awrobl em-sol virm strategy, and acrditional Practice Sheens of stmilar 
problems. 

Outcome tneasuFT . Three learning outcomes, one cogrrftive and two 
-motivational , we>- sessed: (1) performance on a letter series 
criterion test; (<?) causal attributions for performance tn the Letter 
Series Workbook; mc (3) interest in learning more about the letter 
series task. Th< instru||hts developed for each outcome were 
administered as posttest package to each class. (Again, estimates of 
internal consis - ^cy reliability for each measure were satisfactory.) 

Treatment le^finition . Three dimensions of choice over learning 
procedure were -nranipul ated by randomly dividing the subjects in each 
class into a cnoice group (n=84) and a no-choice group (n=81). (Since 
both treatments were implemented in each class, the study actually 
contained nine replications of the same design.) Choice, subjects were 
given their choice of (1). when to do the Letter Series Workbook, time and 
pacing, (2) the sequence in which to complete the workbook, and (3) 
standards for performance on the workbook. No-choice subjects were asked 
to complete the workbook (1) at specified times, (2) in a specified 
order, and (3). according to external ly-def ined performance standards. 

.'«...' ^ / 



These educational choice dimensions were selected as re$r p-sent^axive 
of the actual types of choices presently r^fersd to public school 
students. Yet, the meaning of educational choice can range along a 
continuum from clroice over how to complete a required lessen (as in the 
present study) to choice over whether or mat to attend school. Since 
different psychological processses are likely to be important for 
•different types of ' eduational choices, th€: findings of this study may not 
be general izable to broader choices of alternative learning tasks or of 
whether or not to complete a learning tas*; at all. However, this 
question of general izabil ity is an empirical one, answerable only by a 
systematic investigation of the psycholog :al meaning and learning 
consequences of the full range of choice in education, 

procedure . , The study spanned a four-week period in each classroom, 
During the first week, the experimenter visited each classroom for about 
one hour to conduct the pretesting, introduce the Letter Problems 
Workbook, and divide the students within each class into the choice and 
no-choice groups. (Pilot results had indicated the within class 
treatment manipulation served to strengthen the overall design by making 
the choice/no-choice contrast highly visible.) j 

The. second and thirJ weeks were the workbook phas.. During these two 
weeks, the no-choice group worked on one workbook lesson each day, Monday 
through Thursday of each week, during the 20) minutes the experimenter 
visisted the classroom. No-choice subjects completed the workbook in the 
order in which the lessons were presented. On Friday of each workbook 
week, the experimenter corrected and evaluated no-choice subjects' 
performance on the lessons completed. Choice subjects were instructed to 
complete the workbook before the end of the two week workbook phase. 



The* cauld/frork on the workbooks whenever they had free time during 
sctwcfl and/or when the experimenter visited the classroom to work with 
the wo-dhoice group. Further, they could complete the lessons in an 
jtfrder sr their own choosing. On Friday of each workbook week, when the 
exj*pr inysnter visited the classroom to correct the no-choice group's work, 
chonce subjects were given Answer Sheets and asked to correct and 
evaluate their own work. > 

finally, the three posttesuS were administered during a half hour 
session at the beginning of the fourth week. At the end of this fourth 
week, the experimenter returned to each classroom to follow up on the 
interest posttest. 

Analyses 

Prior to investigating the major ATI hypotheses of the study, factor 
analyses were conducted on both the aptitude scores and the outcome 
scores in an effort to (1) reduce the numbers of variables in the data 
set, (?) avoid the problem of col linearity in multiple regression 
aftJblyses, and (3) seek parsimony in the results. 

A principal components analysis with varimax rotation on 15 aptitude 
scores yielded a six-factor solution, accounting for 75% of the 
variance. The first two factors were labelled G (general ability, but 
more nonverbal and problem-solving ability then verbal ability) and A 
(confidence and belief in one's ability to do well in school). A similar 
principal components analysis on 10 outcome scores yielded four 
orthogonal factors accounting for 72% of the variance. The first of 
these factors was labelled A 1 , representing the belief the ability was 
the cause of performance in the workbook, combined with performance and 
interest learning outcomes. That is, this outcome factor A 1 is parallel 
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to the aptitude factors G and A, in that the cognitive aptitudes that 

loaded on G ancl the affective aptitudes that loaded on A are para^el to 

the cognitive , and affective outcomes that loaded on A 1 . (The remainder 

of the aptitude and outcome fac\or solutions also showed parallels, with 

each outcome factor mapping on to one or more aptitude factors.) 

< 

The major hypotheses of tfie study were tested with a stepwise 

\ 

multiple regression analysis on each outcome factor. In addition, simple 
regression analyses on selected aptitude-outcome pairs (including G-A 1 
and A-/} 1 ) were conducted by treatment across all subjects, by treatment 
within each class, and by treatment between classes (using half-class 
means). The major purpose of these simple regression analyses was to 
explore the marked class differences evidenced throughout the data. 
Given the small n's, the simple regression results were noted as highly 
unstable. Nevertheless, the patterns of these results both highlighted 
the differential effects of treatment on individuals vs. classes, as we'll 
as led to substantive hypotheses about these effects. It is the results 



of these simple regression analyses 

/ 

of this paper. 



for G-A 1 and A-A 1 that are the focus 



Results 

Tables 1 and 2 and Figures 1 and 2 present the results of the simple 
regression analyses for 6-A' and for A-A' by treatment and by treatment 
within class. Figures 3-6 show the corresponding scatter- 
plots. In Figures 1 and 2 the within class lines were plotted to 
represent +1 SD around thk bivariate mean. Given the small. n J s for 
these within class analyses, the results are clearly unstable. For this 
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reason, general patterns of differences, rathe^ than actual R 's or b's 
will be' emphasized. (Classes #3 and #4 were excluded from these within 
class analyses due to extremely small n's.) 

Also .included in Figures 1 and 2 are between class regression lines, 
calculated from half-class means and thus also noted unstable. These 
between class lines were included, however, to investigate possible 
discrepancies between poo^d results based on individuals and class 
results based on class means. As suggested by Cronbach and Webb (1975), 
a large discrepancy would indicate that- the pooled line, assessing 
individual effects, is* probably obscuring large class differences. In 
addition wide variation of the within class lines around the between 

class line is further evidence for large class differences. (Classes #3 

i 

and #4 were included in the calctj 1 atior of the between class lines 
because means for these classes Were generally consistent with the 
pattern formed by the other seven classes.) 

Table 1 

Results of Simple Regression Analyses by Treatment 





No-choice (n=81) 


Choice n=84) 


R 2 b Constant 


R 2 b Constant 


, G-A' 
A-A' 


.19 .46 U.08 
.03 .16 -.1.1 


.30 .52 .07 
.17 .42 .07 
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Table 2 



Results of Simple Regression Analyses by Treatment Within Class 



n 
#2 

#5 
#6 
#7 
#8 
#9 



S-A' 
Class 
Class 
Class 
Class 
Class 
Class 
Class 

A-A' 

Class #1 
Class §2 
Class #5 
Class #6. 
Class #7i 
Class #8 
Class #9 



No-choice 



R2 



10 
10 
10 

9 
12 

9 
11 



10 

w 

12 
9 
11 



.42 
.04 
.43 
.25 
.51 
.84 
.24 



.07 
.03 
.02 
.01 
.49 
.26 
.00 



.74 
.22 
.56 
•.?] 
.70 
..'6 
,75 



.33 
.14 
-.14 
■.07 
.65 
.76 
.04 



Constant 



.12 
-.07 
-.36 
-.52 
.33 
.33 
-.66 



-.03 
.04 

-.35 

•.30 
.78 
.55 

-.44 



Choice 



12 
7- 
12 
II 
"IT 
11 
11 



12 
7 
12 
11 
12 
11 
11 



R2 



.09 
.70 
.42 
.43 
.'21 
.05 
.30 



.06 
.01 
.02 
.72 
.00 
,38 
.10 



.20 
.49 
.63 
.67 
.15 
.29 
.61 



.43 
.13 
.17 
.87 
.02 
.56 
.23 



Constant 



i.ll 
» .23. 
.22 
-.02 
.93 
,-.36 
-.26 



.08 
.48 
.51 
-.54 
1.03 
.10 
-.11 
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; ^ No-choice treatment within class i 
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Note:' Sample range for G was, -2.68 to 2.71 
Sample range for A 1 was -2.47 to 1.82 
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Between class, 
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Figure 1 
Regression Lines for G and A 1 



——Pooled within No-choice treatment 
———Pooled within Choice treatment 

i No-choice treatment within class i 

— i Choice treatment within class i 

Note: Sample range for "A was -3.86 to 1.39 
•Sample range for A' was -2.47 to 1.82 
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Figure 2 
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; Regression Lines for A and A 1 
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Figure 3 

G - A' Scatterplot for No-choice Group 
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Figure 4 

G - A' Seatterplofc for Choice Group 
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Figure 5 

A - A 1 Scatterplot for No-choice Group 
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Figure 6 

A - A' Scatterpldt for Choice Group 
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, Looking first at the results for G-A' , the choice and no-choice 

pooled regression lines are almost parallel, corresponding to the lack of 

significant ATI's in the multiple regression and implying no differential 

effects of treatment on A' for individuals with different levels of G. 

Rather, individuals with higher levels of G performed better than 

individuals with less G in loth the choice and no-choice treatments. 

However, the within class results show a different pattern, the most 

\ • 
striking aspect'of which\is the wide variation among classes. Classes #5 

and #9 show a pattern similar to the overall pooled results, with both 
regression lines essentially parallel, implying no differential effects 
of. treatment. In classes #1, #7, and #8, both lines are positive but t:he< 
no-choice line is steeper. This suggests that in those three classes, \ 
subjects with relatively high G performed substantially better than 
subjects with relatively low G in the norchoice treatment, but only 
slightly better in the choice treatment. Finally, results for classes #2 
and #6 show an ATI pattern in that thechoice line is positive and the. 
no-choice line negative. This ■ suggests that in these two classes, 
subjects with relatively/ high G performed best in the choice treatment 
while subjects with relatively low G performed best in the no-choice 
treatment. , ■ 

.The between class regression lines for G-A' also show a pattern^ 
different from that of the pooled Tines. Although the pooled lines are 
almost parallel, the between class lines are suggestive of a sizeable 
ATI. Such an interaction would indicate that the choice/no-droiEr 
treatment manipulation interacted with classroom\ characteristics 
associated with average G or ability level: students in high (aaierage) 



ability classes benefitted from having choice, while students in low 
(laverage) ability classes learned better with structure. Finally, the 
within class lines show considerable variation around the between class 
lines; further attesting to the existence of large classroom differences 

A-A' 

Similar discrepancies are revealed by V:q a :ets of regression 
lines for A-A 1 . The pooled regression lines do an interaction. 
(This is consistent with the multiple regression results, although the 
Ax treatment interaction accounted for only 2% of the variance.) The 
A-A 1 no-chbice line is almost flat, while the A-A 1 choice line is 
Strongly positive, signifying a much stronger relationship between these 
two variables in the choice cjroup. 

Yet, in none of the classes is the pattern of within class results 
similar to this pattern of pooled results. In classes #8 and #9 both 
lines are positive end essentially parallel, implying that in these two 
classes, students with greater confidence performed better than students 
with less confidence irrespective of treatment. In class #2 both lines 
are also parallel but are negative in slope, again implying no 
differential effect of treatment and a general posttest superiority of 
students low in confidence.' In class #7 both lines are positive but the 
no-ctoice line is ^teeper. In the remaining. three classes (#1,#5, and 
#6), the tv/o regression 1 fries have opposite sldpqs and thus form an ATI. 

\ for A-A 1 the between clas^ regression lines show a considerable 
discrepancy from the pooled regression lines for the "no-choice group but 
not fior the choice group. However, the two between class lines form an 
ATI similar to that shown by the between class lines for.G-A'. 'That is, 
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in relatively high confidence classes, the mean performance of choice 
subjects was considerably greater than that of no- choice subjects. In 
relatively low confidence classes, the pattern was reversed: mean 
performance of no-choice subjects was considerably greater than that of 
choice subjects. In other words, the choice/no-choice treatment 
manipulation interacted with classrtrom characteristics related to 
academic confidence. 

Discussion 

Once again, it should be emphasized that because of limited sample 
sizes the results in this paper are not presented as supportive, evidence 
for a given psychological theory or as conclusive evidence for a given 
learning phenomenon. Rather, they are presented as illustrations of/the 
possible magnitude of social effects in ATI's arising from classroom 
characteristics and as a basis for substantive hypotheses about these 
effects. 

In th§ results for the two selected aptitude- out come pairs (G-A 1 and 
A-A 1 ), the pooled regression lines either shoived no interaction (G-A 1 ) or 
a slight interaction (A-A 1 ) . In practical terms the pooled regression 
lines indicated that students with high ability or high confidence* 
performed better than students with low ability or confidence both when 
given some choice in the learning situation and when the learning 
situation was more structured for. them. 

^Further analyses, however, both within class and between classes 
suggest .that these pooled regression lines were, obscuring large class 
differences. The between class results for b:oth G-A 1 arret A-A' showed 
similar ATI's. In practical terms, these ATI's suggest ttrat students inv 
high ability or high confidence classes learned better when given some 



choice, while students in low ability or low confidence classes learned 
better under more structured conditions. That is, the si gjhiff icant 
interaction was ntrt with individual student aptitude^ hyjt rather with 
the social effects of being a member of a class that was relatively high 
or low in ability or academic confidence. Such social effects might have 
operated as follows. Choice subjects in high ability classes may have 
experienced a mutually felt motivation or challenge and, in low ability 
classes, a common threat, uncertainty^ or abandonment* No-choice 
subjects in high ability classes may. have shared frustration at being 
held.back and, in low ability classes, confidence in not having to make 
all the decisions. This line of interpretation implies that, although 
the major ATI hypotheses of the study were not .confirmed for. individual 
aptitudes, they may have been confirmed for social or classroom 
characteristics. 

The markedly divergent .patterns of within class results provided ., 
additional evidence indicative of large class differences. In some 
classes results suggested that there were no differential effects of 
treatment., In others one treatment seemed to have different effects on 
subjects with different levels of "-ability or confidence, while the other 
treatment did not have differential effects. In still other classes 
treatment interacted with ability or confidence in the formation of an 
ATI. In; the absence of systematic 'assessments of classroom 
characteristics within the experimental design, an informal., post hoc I 
effort was made to find some possible explanations for the widespread 
class differences. This effort included categorizing the different 
patterns of. within class^simple regression results and ranking the nine 
classes on. selected dimensions relating to choice and structure in the / 
classroom. • ^ 
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Resulting from this informal assessment v/ere hypotheses about class 
differences based on the degree to which one or the other experimental 
treatment was similar to everyday classroom experiences. For example, 
several pattern? of results suggested that students high in the given 
aptitude performed best in the treatment most similar to normal classroom 

i procedure, while students low in the aptitude performed best in the 
treatment most different from everyday classroom procedures. 

■ In summary, the results of this study provide tantalizing evidence 
*fcshat significant interactions might r.ot have occurred with individual 
aptitudes but may have occurred with some kind of classroom character- 
istics. Possibly relevant classroom characteristics include the social 

^effects of being a member of a high vs. low ability or confidence class 
and the degree to which the experimental treatments were similar to 
everyday classroom procedures. Future ATI research in this area' would 
benefit enormously from the identification and inclusion of relevant 
classroom characteristics in addition to relevant learner aptitudes. 

Methodological Impl ications 

As Cronbach and Snow (1977) so clearly point out, recognition of the 
importance of classroom effects "forcer a. radical change in thinking 
about ATI" (p. 100). Part' of this change calls for the separation of . 
treatment, between class, and within class effects in analyzing possible 
interactions. , To continue to assume that ATI effects operate 
independently in different individuals is to ignore a priori the 
existence of social effects. Tl^is change in thinking about ATI further 
implies that k in ATI research conducted with classroom groups, classes 
rather than individuals are the proper unit of analysis for statistical 



inference. Furthermore, following from Cronbach and Snoy/'s earlier rule 
of thumb, samples of 100 classes per treatment are needed! to detect 
aptitude main effects or ATI's with sufficient power. In\full 
recognition that such sample sizes are well beyond the resources of most 
researchers, Cronbach and Snow advise ATI researchers to rely more 
heavily oh "the theoretical coherence of the results' 1 (p. 104) than on 
the statistical significance of the results. 

The results of this study provide an additional methodological 
suggestion that may be of value to some ATI researchers. Unlike most 
studies, in which whole cl asses' are aszigned to one treatment, all 
classes in this study were exposed to both treatments. This component of 
the study v/as designed deliberately to heighten the contrasts betv/een the 
choice and no-choice treaments, which in actuality, were quite small. 
This design component further allowed for the analysis of within class 
effects of both treatments, rather than just one. The within class 
analyses therefore consisted of nine mini-ATI's, all paralleling the 
pooled within treatment ATI analysis. With this procedure the class x 
aptitude and class x aptitude x treatment interactions are 
distinguishable. 

The major limitation of this kind of design remains, of course, the 
limited sample sizes for the within class analyses; , Nonetheless, the 
designed implementation of two experimental treatments in each classroom 
'group may be. a valuable methodological strategy for ATI researchers 
interested in (1) strengthening the treatment contrast and/or (2) 
assessing ATT s for different kinds ofylearners within the same class and 
for the same* kinds .of TearrieVs In; different types of classes. 
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